Decoding message data embedded in an image print via halftone dot orientation

ABSTRACT

What is disclosed is a novel system and method for encoding/decoding data in a cover contone image via halftone dot orientation modulation. Arrays of halftone threshold values are used to determine a desired orientation, e.g. 0/90°+/−45° for a given single data value of the original message to be embedded. Message data is embedded as a function of halftone dot orientation. Detection modeling of the print-scan process enables the determination of dot orientation from the image scan via statistically motivated image moments. A probabilistic model of the print-scan channel conditions received moments on input orientation. Density values of the received moments are used to determine dot orientation for each halftone cell. The embedded data is retrieved based on the determined orientations. The present method is applicable to areas of data embedding, document security, and the like.

CROSS REFERENCE TO RELATED APPLICATION

The present patent application is related to commonly owned U.S. patentapplication Ser. No. 12/207,704 entitled “Encoding Message Data In ACover Contone Image Via Halftone Dot Orientation”, filed on the samedate herewith, the entire teachings of which being hereby incorporatedby reference in its entirety.

TECHNICAL FIELD

The present invention is directed to systems and methods forwatermarking which embed (encode) message data in an image and whichextract (decode) the embedded message data from the scanned image print.

BACKGROUND

Watermarking is the process of embedding information into a signalwithout impacting its primary functionality. The signal may be audio,pictures or video, for example. Image watermarks are further classifiedas perceptible (visible to the naked eye in case of images) orimperceptible, and the particular choice is motivated by theapplication. See generally: Digital Watermarking, by: I. J. Cox, M. L.Miller, and J. A. Bloom, Morgan Kaufmann, (2001). Multimedia data hidinghas been an active area of research with several emerging applicationsin content authentication, anti-piracy data hiding, and digital rightsmanagement.

Hardcopy watermarking or data hiding in images that is required tosurvive the print-scan (digital-analog-digital) process has come toreceive significant attention recently. The fact that printed mediacomprises an increasing portion of official, legal and transactionaldata such as IDs, passports, contracts, and the like, makes the problemof considerable economic interest. Particular motivating applicationslie in identifying where a hardcopy document originated, documentintegrity verification and copy protection for ensuring that copies ofan original hardcopy are distinguishable and identifiable as such,copyright enforcement methods for identification of copyright ownership,and auxiliary data embedding such as web URLs, product UPC codes, etc.Hardcopy data hiding can be distinguished from other multimedia datahiding schemes by the presence of a print-scan process. Hardcopy datahiding schemes must adapt to the characteristics of the print-scandistortion channel.

One key component of the printing process is a bit-depth reduction stepcalled digital halftoning which reduces the original continuous tone,typically 8 bits/pixel, image to a 1 bit/pixel binary image. Halftoningaims to produce an illusion of continuous tone by trading off amplituderesolution for spatial resolution. Algorithms for digital halftoning canbe generally classified into three categories: 1) point processes, orscreening methods; 2) neighborhood processes or error diffusion; and 3)iterative or search based methods. Of the above, screening (particularlyclustered-dot) is dominant in xerographic printers, while errordiffusion is used rather extensively in inkjet printers. Once printed,the image transitions to the analog domain, and can be re-digitized byscanning. For hardcopy data hiding, it should be noted that the input tothe physical print-scan channel is a binary or halftone image and hencethe distortions are tied to the nature of the halftoning algorithmemployed.

Methods for data embedding in hardcopy images may be grouped into twocategories. The first corresponds to robust embedding methods that areintended to survive printing and scanning but do not directly exploitcharacteristics of the printing process during embedding. The secondcorresponds to techniques that use the particular characteristics of theprinting process (e.g. halftoning) for embedding. The exploitation ofthis specific knowledge typically offers greater potential for embeddingand is better suited to hardcopy applications since the embedding occursjust prior to printing. The methods in the first category are limited bythe lack of knowledge of the halftone print process. From acommunications viewpoint, this adversely affects detection accuracybecause the embedding does not adapt to the characteristics of theprint-scan channel. Methods in the latter category require a manualvisual detection/inspection process, e.g. by overlaying a pre-designedpattern directly over the hardcopy print. While this is useful for someapplications such as document authentication, a large class ofapplications such as meta-data embedding, document tracking inworkflows, and secure hiding in adversarial scenarios, are betterenabled by automated data recovery. Methods which allow automateddetection tend to perform embedding by modulating binary halftoneoutputs with pre-determined message signals to produce the watermarkedimage containing the embedded data. This can cause distortions in theoutput image even for modest embedding rate unless sufficient care istaken in the design of the embedding method.

Hardcopy data embedding techniques that allow automated extraction ofthe embedded data without involving a human observer are a key enablerfor a wide variety of document security applications. Methods developedfor this purpose can be categorized as data encoding or data hidingapproaches. For methods in the former category, the data is embedded ina region of the printed page that is solely dedicated to the objectiveof conveying the information, the visual appearance of the encodedregion being only a secondary concern. One and two-dimensional bar-codesare the predominant representatives of techniques in this class.Adaptations of these methods are also utilized for limited imagerendering applications.

Methods for clustered-dot halftone pose significant challenges for datahiding methods. The requirement for a faithful reproduction of theoriginal cover image can have irreconcilable conflicts with the abilityto introduce a distinguishable change in the printed image for thepurpose of embedding data. As a specific example, in regions where thecover image is white, the desired rendering is white (regions free ofany printed dots), in which situation no discernible change can beincorporated. A similar conflict arises in regions where the cover imageis black. In these scenarios, as is desirable for data hiding methods,embedding robustness must be relaxed in favor of cover image fidelity.Methods for data hiding in hardcopy image prints must carry the embeddeddata in a manner that minimally disrupts image content while stillallowing data recovery at the decoding end. These can be challengingconstraints.

Accordingly, what is needed in this art are increasingly sophisticatedsystems and methods for high capacity binarized message data embedding(and decoding) in a contone image which addresses the above-discussedproblems in this art while achieving high embedding data rates.

BRIEF SUMMARY

What is provided are a novel system, method, and computer programproduct for encoding data in a contone image using halftone dotorientation. Analytic halftone threshold functions are used to producescreening-based clustered halftone dots for a given gray level in theimage. The host signal, i.e. the gray level, is used in the mapping fromthe data message (binary symbol) to the embedded mark (dot orientation)space. Each unique state which a single data value of the original datamessage to be embedded is associated with a particular dot orientationdefined with respect to a major axis of the elliptical dot, e.g.horizontal (0°), vertical (90°), diagonal (+/−45°), etc. Halftone dotsin each uniform periodic tile of the contone image are generated using ahalftone threshold array that has an orientation determined by theunique state of the single data value embedded in that cell. The imagecontaining the oriented halftone dots is then rendered to an outputdevice. On the decoding end, the image is scanned. Following local andglobal synchronizations which address geometric distortions introducedby the print-scan channel, orientation of each halftone dot of eachperiodic tile of the scanned image is estimated via statisticallymotivated image moments using a probabilistic model of the print-scanchannel which specifically conditions received moments on inputorientations. Error control coding is optionally incorporated in orderto correct for errors introduced in this detection process. For eachhalftone cell, the computed image moments are utilized either: a)directly in order to estimate the dot orientation for the cell andthereby the single data value associated with the associated dotorientation or b) along with a probability density model for the imagemoments conditioned on the input orientations that provides a likelihoodvalues for the possible dot orientations. The decoding component of theerror control coding then estimates the embedded message from thecollection of the individual single data values or their likelihoodvalues. Given the inherent challenges in halftone data hiding, errorcontrol coding is inevitably necessary for most applications though, inlimited cases, this step may be unnecessary and may be dropped.

In one example embodiment of the present method for data embedding viahalftone dot orientation, a cover contone image having a plurality ofcontone data points comprising pixel intensity values is received. Thedata message to be embedded in the cover contone image comprises aplurality of single data values. Further, each single data value assumesonly one unique state. Each of the unique states is associated with aunique orientation direction. The received image is partitioned into aperiodic tiling. An array of threshold values is defined for each uniquestate and is designed to produce the associated dot orientation. Each ofthe threshold values in the threshold array is defined for a givenscreen frequency and dot shape. Each array of threshold values has aone-to-one correspondence with pixel intensity values encompassed byeach periodic tile. A unique periodic tile wherein a single data valueis to be encoded is identified for each data value of the originalmessage to be embedded. For each identified tile wherein a single datavalue is to be encoded, each pixel intensity value in the current tileis compared with a corresponding value of the threshold array definedfor the unique state of the single data value to be encoded in thecurrent tile. Each comparison produces a binary output value for eachpixel in the current tile. In one example, a binary output value of 1defines a pixel being turned “ON” at that location in the current tileand a binary output value of 0 defines a pixel being turned “OFF” atthat location. Turning pixels within a single cell ON/OFF enables thehalftone dot of that cell to be printed with a particular orientationfor the elliptical dot. All of the comparisons for the current tileproduces a binary mask for that tile. All of the binary masks for alltiles produces a single mask for the image. The image is renderedaccording to the mask with each halftone dot of each tile assuming thedesired dot orientation defined by that tile's binary mask. The outputimage thereafter contains the embedded original data message.Embodiments regarding error correction detection are disclosed.

In one example embodiment of the present method for decoding dataembedded in an image via the above-described method of dot orientation,a printed image containing the embedded original message is scanned. Themessage encoded in the image comprises a plurality of single datavalues. Each single data value can assume only one of a plurality ofpossible unique states; each of the possible states is associated with aunique orientation direction with respect to a horizontal and verticalaxis of the image. The scanned image is synchronized to compensate forgeometric distortions introduced by the scan process. In one embodiment,the image is synchronized based on estimated global rotation and scalingfactors computed from the estimated periodicity in the scanned halftoneand from local estimates of variations in this periodicity. Each singledata value of the message embedded in the scanned image print has onlyone unique state and each of the unique states is associated with aunique dot orientation as was used for encoding. The scanned image ispartitioned into periodic tiles. For each of the periodic tiles in thescanned image, a plurality of image moments are calculated for the dotprinted in the current tile. One image moment is calculated along eachof the associated unique orientation directions. Embodiments forcalculating image moments (σ_(x), σ_(y)) for orientation directionsalong orthogonal x and y axis based on a computed center of mass for thehalftone dot are provided. A largest of the calculated moments is usedto identify an orientation direction for the dot in the current tile.The single data value in the current tile is decoded based on the uniquestate associated with the dot's determined predominant orientationdirection. The original message embedded in the image print isreassembled from each unique state of each decoded single data value.Error correction can be performed on the decoded message oralternatively the estimated moments for the tiles can be collectively inan error correction decoding procedure to obtain an estimate of themessage.

The foregoing and other features and advantages will be apparent fromthe following more particular description of the various embodiments ofthe invention, as illustrated in the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other features and advantages of the subject matterdisclosed herein will be made apparent from the following detaileddescription taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 illustrates one example single halftone cell or “periodic tile”(not to scale) whereon a halftone dot has been printed, each cellencompasses a plurality a pixel locations as defined by the device'shalftone screen;

FIG. 2 illustrates example orientation directions in accordance with theteachings hereof;

FIG. 3 broadly illustrates one example block diagram for embeddingmessage data in a contone image via dot orientation modulation;

FIG. 4 illustrates a flow diagram of one example embodiment of thepresent method for data embedding via dot orientation modulation;

FIG. 5 illustrates an example gray level halftone image wherein messagedata has been encoded in accordance with the present method of dataencoding via halftone dot orientation modulation;

FIG. 6 illustrates an example array of pixel intensity values for thecell of FIG. 2A;

FIG. 7 illustrates an example threshold array T₁(x,y) to be compared tothe array of pixel intensity values of FIG. 6;

FIG. 8 illustrates an example binary mask for the cell of FIG. 2Aresulting from the comparison of the intensity values of FIG. 6 witheach threshold value in the threshold array of FIG. 7;

FIG. 9 broadly illustrates one example block diagram for decoding datafrom a scanned image which was embedded using the present halftone dotorientation method;

FIG. 10 illustrates a flow diagram of one example embodiment of thepresent method for decoding data which has been encoded via halftone dotorientation;

FIG. 11A shows histograms of the received moments, σ_(x), σ_(y), for the(transmitted) horizontal orientation ⊖₁ given two orientations at a graylevel of 32%;

FIG. 11B shows the histograms of the received moments, σ_(x), σ_(y), forthe vertical orientation ⊖₂ given two orientations at a gray level of32%;

FIGS. 11C-D repeat the histograms of FIGS. 11A and 11B for a gray levelof 52%;

FIG. 12 plots bit error rate (BER) performance across gray levels;

FIG. 13 plots bit error rate (BER) performance as a function of the coderate;

FIGS. 14A-E illustrate the effects of data embedding on a sample coverimage;

FIG. 15 illustrates one example document reproduction system whereinvarious aspects of the present method are likely to find their intendeduses; and

FIG. 16 is an explanatory diagram illustrating one example of a computerreadable storage medium capable of storing machine readable instructionswhich, when mounted on a computer system, cause the computer system toperform one or more aspects of the present method.

DETAILED DESCRIPTION

What is disclosed is a system and method for data embedding via halftonedot orientation modulation. Data is encoded in a cover contone image asa function of halftone dot orientation modulation. A method for decodingmessage data embedded in an image print via halftone dot orientation isalso disclosed.

It should be understood that one of ordinary skill in this art shouldclearly understand many facets of halftone processes, halftone screens,halftoning, and other related techniques common in the color sciencearts. Additionally, one should also be readily familiar withmathematical techniques involving the computation of image moments,conditional densities, probabilistic modeling, and the like, asdiscussed herein. One of ordinary skill would also be knowledgeableabout computer science and hardware and software systems and algorithmssufficient to implement the functionality and capabilities describedherein in their own document reproduction environments without undueexperimentation.

An “image” refers to a pattern of physical light comprised of knowncolors of the light spectrum which are visible by the human eye. Animage input device receives the image. Image input devices includescanners, cameras, photography equipment, copiers, facsimile devices,photo reproduction equipment, and the like. To render an image is tooutput the image to an image output device. Image output devices receivethe image signal and reduce the signal to viewable form. Example imageoutput devices capable of image reduction include printers, copiers,monitors, projectors, and the like. Image reduction includes the processof marking a substrate with colorants to form the image from the visualintegration of the colorants on the substrate, or to display the imageon an image display device.

A scanner is an image input device that optically scans images, printmedia, and the like, and converts the scanned image into a digitizedformat. Common examples are variations of the flatbed scanner known inthe arts wherein specialized image receptors move beneath a platen andscan the media placed on the platen. Modern scanners typically use acharge-coupled device (CCD) or a contact image sensor (CIS) as the imagesensing receptor. A signal of the scanned media is produced by thescanning device. Such an image signal contains digitized informationabout pixels and their locations within the scanned print.

Halftoning is the process of representing a continuous tone (contone)image by a bi-level image such that, when viewed from a suitabledistance, the bi-level image gives the same impression as the contoneimage. Halftoning reduces the number of quantization levels per pixel ina digital image. Over the long history of halftoning, a number ofhalftoning techniques have been developed which are adapted fordifferent applications. Due to their stability and predictability,clustered dot halftoning is one of the primary choices for xerographicprinting systems. Traditional clustered dot halftones were restricted toa single frequency because they were generated using periodic gratingsthat could not be readily varied spatially. Halftoning techniques arewidely employed in the printing and display of digital images and arenecessary because the physical processes involved are binary in natureor because the processes being used have been restricted to binaryoperation for reasons of cost, speed, memory, or stability in thepresence of process fluctuations. Classical halftone screening applies amask of threshold values to each color of the multi-bit image.Thresholds are stored as a matrix in a repetitive pattern. Each tile ofthe repetitive pattern of the matrix is a halftone cell. Typically,pixels are converted to black if they are below the threshold and whiteotherwise. Digital halftones generated using threshold arrays that tilethe image plane were originally designed to be periodic for simplicityand to minimize memory requirements. With the increase in computationalpower and memory, these constraints become less stringent.

As used herein, the term “unique state” refers to one of the states froma predefined set. For example, in a 2-state system (two orientationdirections), the set of binary values is: {0, 1}. This set contains atotal of 2¹=2 possible unique states a single data value can take. In a4-state system (4 orientation directions), the set of binary values is:{00, 01, 10, 11}. This set contains a total of 2²=4 possible uniquestates a single data value can take. Another set contains the binaryvalues: {000, 001, 010, 011, 100, 101, 110, 111}. This set contains atotal of 2³=8 unique states, and so on. Note, while sets of binaryvalues are shown for ease of illustration, the unique states can ingeneral be derived from any arbitrary set whose cardinality does notnecessarily has to be a power of 2.

A “data message” or “original message”, as used herein, refers to anyinformation (text, numeric, etc.) which can be reduced to a binarizedbit stream of 1's and 0's representative of a collection of uniquestates. An original message is composed of a plurality of single datavalues. Each of the single data values is associated with one of thepossible unique states taken from one of the sets of binary values, asdefined above, used for encoding. An original data message having a dataformat other than binary must be converted to a binary bit stream priorto embedding. Any of a plurality of techniques well known in the artscan effectuate data conversions. The unique state of each single datavalue retrieved from each periodic tile during decoding collectivelyforms the original message.

Single data values are encoded in the image by orienting halftone dotsin a unique orientation direction. As will be discussed in furtherdetail herein, each of the unique states from a given set of binaryvalues used for encoding is associated with a unique orientationdirection. If a 2-state system is used for encoding then a single datavalue can assume one unique state (0 or 1) from the defined set ofbinary values. In this case, two unique orientation directions arenecessary in order to embed all possible single data values. Forexample, the unique state of 0 might be associated with an orientationdirection along a major x-axis (horizontal 0°). The unique state of 1would then be associated with an orientation direction orthogonal to thex-axis along the major y-axis (vertical 90°). In a 4-state system {00,01, 10, 11}, a total of 4 unique orientation directions would be neededto embed all possible single data values. In this instance, the uniquestate of 00 might be associated with an orientation direction along themajor x-axis; the unique state of 01 might be associated with the majory-axis; the unique state 10 might be associated with the diagonal axis(+/−45°); and the unique state 11 associated with yet anotherdiscernable orientation direction.

At the onset hereof, it is important to provide a brief explanatoryillustration which will provide a basis for an understanding of theteachings hereof.

Reference is made to FIG. 1 which illustrates one halftone cell 100whereon a halftone dot 102 has been printed. Each halftone cellencompasses a plurality a pixels. Each pixel 101 has its own respectivelocation within a cell. Cell size for a given halftone screen isconstant. It should be clear that a halftone cell may encompass more orfewer pixels than are being shown for explanatory purposes. Thus, thecell of FIG. 1 is clearly illustrative. Printed dot 102 is alsoillustrative. In practice, a dot would be printed at a resolution suchthat the jagged lines shown were not visible to the eye. Cell 100represents a very small region of a larger region on a print mediawhereon all halftone dots are printed. The collection of printedhalftone dots forms the viewable image. The resolution of the imageoutput device is usually defined by the number of pixels. The dimensionsof the halftone screen designed for a particular marking device definesthe dimensions of the halftone cells (or tiles).

The fine resolution in which pixels are printed visually integratesneighboring pixels. The visual integration of pixels creates a specificgray level or color for the printed dot, for example, light gray, darkgray, sky blue, azure green, or one of millions of possible colors. Themixing of differing colors creates a specific color defined by thedesigners of the color marking device. Neighboring cells become visuallyintegrates to collectively form, for example, a blue sky or a azure seaof an image. Not all pixel locations of all cells have colorantdeposited thereon. For each tile, a matrix defines whether a pixel isturned ON or OFF. If the value of the mask at a location is “ON” then,in one instance, a colorant is deposited at that pixel location. If thevalue of the mask at a location is “OFF” then no colorant is depositedat that location. Information about what has been printed at each pixellocation can be retrieved from a scan of the image using an imagescanning device. The image sensors of the scanning device providedigitized information about at each pixel. Preferably, the image isscanned with a scanning device having at least the same resolution or ahigher resolution (number of pixels) as the color marking device used tooutput the image print.

Reference is now made to FIGS. 2A-B which illustrate example orientationdirections in accordance with the teachings hereof. The halftone dot ofFIG. 1 is shown reprinted in cell 200 of FIG. 2A. Dot 201 is the muchsmaller dot 202 shown substantially enlarged to illustrate theapproximately elliptical dot having been oriented in a directionsubstantially along the y-axis 203. The cell 204 of FIG. 2B shows dot205 substantially enlarged from dot 206 to illustrate the approximatelyelliptical dot oriented in a direction substantially along the x-axis207. The elliptical dot can be additionally be oriented substantiallyalong a diagonal axis (+/−45°) (not shown). Another orientation could berepresented by the dot being printed substantially circular and thus notoriented along either coordinate axis.

As previously discussed, each of the unique orientation directions ofthe halftone dots is associated with a unique state. In a 2-state systemhaving {0, 1}, the state of 0 might be associated for instance with thedot orientation direction of FIG. 2A and the unique state of 1 might beassociated with the dot orientation of FIG. 2B (or vice-versa). Eachsingle data value of the message having a unique state of 1, forinstance, is printed in its respective cell with an elliptical dotelongated in the direction of the x-axis. Single data values with aunique state of 0 are printed in their respective cells with anelliptical dot elongated in the direction of the y-axis. In decoding,dots determined to have an orientation predominantly along the y-axiswould be decoded as having unique state 0. Likewise, dots determined tohave an orientation predominantly along the x-axis would be decoded ashaving unique state 1.

In a 3-state system, having three unique orientation directions, a dotdetermined to have, for example, an orientation direction predominantlyalong the x-axis would be decoded as unique state 00. A dot determinedto have an orientation direction predominantly along the y-axis would bedecoded as unique state 01. A dot determined to have an orientationdirection predominantly along a diagonal axis would be decoded as uniquestate 10 (or alternatively unique state 11 since only three orientationdirections are being used). Hence, no 4^(th) orientation direction isneeded. In this example, the single data values can only assume of threeof the four possible unique states from the set: {00, 01, 10, 11 }.

In the present method, data is embedded in a cover contone image as afunction of dot orientation. Each of the unique states that a singledata value can have is mapped to a respective dot orientation direction.Each unique state is associated with an array of threshold valuesdesigned to produce a particular dot orientation. Pixel intensity valuesof each periodic cell wherein a single data value is to be embedded arecompared to the corresponding values of the associated threshold array.The result of all the comparisons for all pixel locations within a giventile produces an array of 1's and 0's (binary mask) for that tile.Binary values of a mask for a given tile define whether a given pixelwithin the tile is turned ON or OFF. Turning pixels ON/OFF enables thedot to be printed in an ellipsoid shape having the intended orientatione.g. horizontal (0°), vertical (90°), diagonal (+/−45°), etc. The dotorientation is associated with the unique state of the single data valueembedded in that tile. The image is printed with the halftone dotsoriented in the intended directions defined by the single data valuesembedded therein. Given the high resolutions of today's printingdevices, slightly elongated ellipsoid dots oriented in one direction oranother are too small to be seen by the unaided eye. Since most imagescontain many halftone tiles, a huge number of single data values of anoriginal message can be encoded in an image. One implementation hereofdemonstrates that up to 3×10⁵ bits can be embedded and recovered from an8″×8″ image scan. The encoded message is retrieved following global andlocal synchronizations which address geometric distortions introduced bythe printing system utilized for the image print and in the scanningprocess for capturing a digital image from the image print. Imagemoments are calculated for each halftone dot, in a manner as discussedherein further. A maximum-likelihood function uses density values of theimage moments to determine orientation direction. The unique stateassociated with the determined dot orientation direction decodes thesingle data value encoded in that cell. The original message isreconstructed from the collection of decoded single data values.

Reference is now made to FIG. 3 which broadly illustrates one exampleblock diagram for embedding a data message in a contone image via dotorientation modulation.

A cover contone image 302, denoted I(x,y), is scanned into digitizedform using scanning device 304. A plurality of contone data points isobtained. A data message 306 is received by a channel encoder 308 whichintroduces redundancy in the data to enable error recovery at thedecoding end. The decoder produces a data stream of coded data 310.Using the cover contone image and the coded data bits, single datavalues are encoded via dot orientation modulation 312. The image ishalftoned using halftone dots in the determined orientations required toembed the coded data. Data embedding and halftoning are preferablypreformed concurrently where feasible. If data is encoded prior tohalftoning, it operates on the contone image and the image may bedistorted. If data embedding is performed after halftoning, it isnecessary to ensure that the data embedding process retains imagequality, which can be challenging since the image is already in abinarized representation. The resulting halftone image 314, denotedI^(h)(x,y), is printed on marking device 316 and hardcopy 318 of theimage containing the embedded original message is produced. It isunderstood that the cover contone image may be alternatively obtainedfrom a different digital capture device such as a digital camera, slidedigitizer, etc.

Reference is now made to FIG. 4 which illustrates a flow diagram of oneexample embodiment of the present method for data embedding via dotorientation modulation.

At 402, a cover contone image comprising a plurality of contone datapoints is received. The image is received through a scanning device.Individual pixels are represented in a digitized form. Each of thedigitized values represents an intensity value at a given pixellocation. An original message composed of a plurality of single datavalues is to be embedded in the received image. Each single data valuecan assume only one unique state from the set of binary values definedfor the encoding process. The set of binary values is defined by thetotal number of unique orientation directions. Each of the orientationdirection is associated with one of the possible unique states anysingle data value of the original message can take.

At 404, a periodic tiling is defined for the image. The uniform tilingis based on a spatial extent of the image as defined by the halftonescreen. Each of the uniform periodic tiles encompasses an array of pixelintensity values. Each uniform tile is a single cell containing a singledot. No more than one dot is printed in any given cell.

At 406, for each unique state a single data value can assume, an arrayof threshold values is defined. Individual threshold values are definedfor a given halftone screen frequency (number of pixels arrayed in asingle cell) and a given dot shape (unique orientation direction). Thenumber of threshold values corresponds to the number of intensity valuesarrayed in each tile. In other words, there exists a one-to-onecorrespondence of pixel intensity values in each cell to thresholdvalues in each threshold array. In a two-state system wherein a singledata value can assume one of two unique states (0 or 1), two thresholdarrays are defined, one for each unique state. One threshold arrayT₀(x,y) would be associated with the unique state of 0. Anotherthreshold array T₁(x,y) would be associated with the unique state of 1.Each threshold array is designed to produce a resulting dot with anelliptical shape elongated in the unique orientation directionassociated with the single data value's unique state. The orientationdirection is thus representative of the unique state of the single datavalue embedded in that tile.

At 408, each single data value is assigned to a specific tile in theimage wherein it is to be encoded. Data can be encoded in tiles in acertain region of the image as it may be desirable to embed the originalmessage only in a defined region within the image such as, for instance,an upper corner or in a center portion, or alternatively in an object inthe image or in specific image content. In this instance, periodic tilesin regions wherein the original message is not embedded can eithercontain no single data values (dots not elongated in any direction), orcan contain a predetermined sequence of bits such as all 1's or all 0's.Defined bit sequences may be added (or padded) to the original messagesuch that every tile in the image has a single data value embeddedtherein. This may be done in instances where all tiles of the entireimage are preferably processed at a single time instead of only certaintiles in defined image regions being processed separately. Bits or bitsequences used as padding can be readily identified during decoding andremoved from the retrieved bit stream when the original message isreassembled.

At 410, for each pixel in each periodic tile, the intensity value atthat pixel location is compared against its corresponding thresholdvalue of the threshold array defined for the single data value to beencoded in the current tile. For instance, in a two-state system, if thesingle data value to be encoded in the current tile has a unique stateof 0, the threshold array T₀(x,y) is used for comparison purposes.Threshold array T₀(x,y) is designed to elongate the elliptical dot in anorientation direction associated with the unique state of 0 (such ashorizontal). If, on the other hand, the single data value to be encodedin the current tile has a unique state of 1, then the threshold arrayT₁(x,y) is used for comparison purposes. Threshold array T₁(x,y) isdesigned to elongate the elliptical dot in the orientation directionassociated with the unique state of 1 (such as vertical).

At 412, the image is then halftoned using the binary masks. Printing ahalftone dot according to the binary mask for a given cell elongates thehalftone dot along the intended orientation direction according to theunique state of the single data value encoded in that tile. The imagecontaining the embedded message can then be rendered to an outputdevice. The message is secure because it can only be decoded from theimage wherein it has been embedded through an understanding of thevarious unique states the single data values assumed and the orientationdirection associated with each of the unique states.

The description which follows focuses on binary data using twoorthogonal orientations, e.g., the x-axis in a first direction(horizontal), and the y-axis in a second direction (vertical). Fornotational simplicity, the following is based on a 0°/90° case wherein aunique state of 0 corresponds to an orientation direction along a majorx-axis and a unique state of 1 corresponds to an orientation directionalong a major y-axis.

Reference is now made to FIG. 5 illustrating an example gray levelhalftone image wherein message has been embedded in accordance with thepresent method of data encoding via halftone dot orientation modulation.Image subsection 503 represents a small section of image 502 having onemajor axis in the x-direction (horizontal) and the other major axis inthe y-direction (vertical). This orientation represents the 0°/90° case.Image subsection 504 represents a section of image subsection 503. Imagesubsection 504 illustrates the individual halftone dots and theirrespective orientations. The respective orientation directions of eachof the halftone dots of image subsection 504 correspond to the uniquestate of the single data value encoded in each respective halftone cell.Halftone dot 505 of image subsection 504 has an elliptical dot orientedsubstantially in a horizontal orientation along the x-axis. Halftone dot506 has an elliptical dot oriented substantially in a verticalorientation along the y-axis. As previously discussed, the orientationof each of dot is based on the unique state of the single data valueencoded in that cell.

Reference is now made to FIG. 6 illustrating an example array 600 ofpixel intensity values for the cell of FIG. 2A. Each intensity value inthe array will be compared to a threshold value in the correspondingthreshold array associated with the unique state of the single datavalue to be encoded in this cell. Per this example, the unique state of1 corresponds to a dot orientation in a direction along the major y-axis(as shown in FIG. 2A). The respective threshold array for this cell isassociated with the unique state of 1. The threshold array is designedto produce a binary mask which orients a halftone dot along the y-axis.

Attention is directed to FIG. 7 illustrating an example threshold arrayT₁(x,y). The individual threshold values of the threshold array arecompared to the pixel intensity values of FIG. 6. Threshold array 700 isassociated with the unique state of 1. The single data value to beencoded in this particular tile has a unique state of 1. This thresholdarray is designed to orient a dot in a direction along the y-axis. Inthis example, halftone dots oriented along the y-axis encode a singledata value having a unique state of 1. All halftone dots oriented alongthe x-axis encode a single data value having a unique state of 0 usingthreshold array T₀(x,y) associated with this particular unique state. Itis understood that the threshold array T₁(x,y) in FIG. 7 is utilized forease of illustration here where the values in the threshold array areeither 0 or 40. Actual threshold arrays would have entries distributedover a larger set of possibilities in order to allow good representationof different gray levels in the output halftone image.

As previously discussed, depending on the unique state of the singledata value to be encoded within any give periodic tile, a thresholdarray T(x,y) is used for comparison purposes. Each threshold array isdesigned to produce a unique dot orientation.

One example threshold function used to modulate dot orientations alongthe x and y axis, is given by:T(x,y)=sgn(cos(2πf _(x) X)cos(2πf _(y) Y))|cos(2πf _(x) X)|^(Yx)|cos(2πf_(x) Y)|^(Yx)

where f_(x) and f_(y) are parameters that control the dot shape in the xand y directions.

Programmable parameters, γ_(x) and γ_(y) are used for dot orientation.Setting one of these parameters to a larger value gives rise toelliptically shaped dots oriented along one of the vertical orhorizontal directions. For instance, if γ_(x) is greater than γ_(y), thedot is oriented along a vertical axis. If γ_(y) is greater γ_(x) thanthen the dot is oriented along a horizontal axis. In the case whereγ_(x) equals γ_(x), the dot is symmetric about the x and y axis. Thefunction sgn ( ), which extracts the sign of a real number, is given by:

${{sgn}(t)} = \{ \begin{matrix}1 & {{{if}\mspace{14mu} t} \geq 0} \\0 & {{{if}\mspace{14mu} t} = 0} \\{- 1} & {{{if}\mspace{14mu} t} < 0}\end{matrix} $

Dots are oriented based on the following relationship:

${I^{0}( {x,y} )} = \{ \begin{matrix}1 & {{{{if}\mspace{14mu}{I( {x,y} )}} \geq {T( {x,y} )}},} \\0 & {{{{{if}\mspace{14mu}{I( {x,y} )}} < {T( {x,y} )}},}\mspace{14mu}}\end{matrix} $

Using the above relationship, if the intensity value at a given x,ypixel location within a tile is greater than or equal to thecorresponding threshold value of the associated threshold array then abinary output value of 1 is produced (pixel is turned ON) for that pixellocation. If the pixel intensity value is less than the correspondingthreshold value then the binary output value is 0 (pixel is turned OFF)for that pixel. The collection of comparison results for a given tileproduces a binary mask of 1's and 0's for that tile. Printing the dotaccording to the cell's binary mask produces a dot with the orientationdirection associated with the unique state of the single data valueencoded in that cell.

A comparison of each of the intensity values in the array of FIG. 6 withtheir respective threshold values in the threshold array of FIG. 7proceeds according to the threshold function 710. A binary output value(0 or 1) is produced for each pixel location as a result of thecomparison.

FIG. 8 illustrates an example binary mask for the cell of FIG. 2Aresulting from the comparison of the intensity values of FIG. 6 witheach of their respective corresponding threshold values in the thresholdarray of FIG. 7. In the example illustration, pixels are printed in eachlocation where the binary mask has a value is 1 and pixels are notprinted where the value is 0.

The present method for data embedding via halftone dot orientationmodulation provides many advantages. In instances where it is desirableto embed multiple original messages in a single image, a first messagecan be encoded in one region of the image and a second message encodedin another region. In the remaining regions not containing the originaldata message, identifiable bit sequences can be used as “filler”. Suchbit sequences may be, for example, a repeating sequence of “100100100”or “100010001”, such that bit sequences decoded from regions of theimage wherein real message data is not embedded can be readilyidentified during decoding and removed. Alternatively, a “tag” can beadded to the beginning and end of each message to identify, duringdecoding, when the message starts and when the message ends. Datasegments decoded not having the appropriate tag would be ignored.

Optionally, a tag can be embedded in a specific location within theimage such as, for instance, the lower left corner or upper right, toidentify the specific bit sequence “filler” or “padding” used in theencoding process. Extraneous bits can be readily separated from the realmessage. The tag could be retrieved at the start of the decoding processfrom a location in the image where such a tag is known to be embedded,and used to facilitate the removal of identifiable bits or bit sequencesfrom the decoded data bit stream. Removing padding or filler bits fromthe data message can be done during decoding but may alternatively bepreformed in a post-processing step wherein the image may be furthermanipulated, converted, error corrected, translated, and the like.

Since the image itself has an x, y axis, it can be thought of as amatrix of periodic tiles. Embedded tags can alternatively be utilized toidentify a starting tile in the matrix of periodic tiles within theimage wherein the message is to be found. For example, a tag decodedfrom the lower left corner of the image might define a tile location(150, 875) indicating a starting tile wherein a first single data valueof the original message can be found. A second tag retrieved from, forexample, the upper right corner might be used to define a secondlocation where a second message can be found. A second tag might alsodefine a tile location where the embedded message ends. Tags, filler,and bits used as filler can each be embedded in the image using theteachings hereof. Many additional embodiments, features, andenhancements are envisioned and are intended to fall within the scope ofthe appended claims.

Tags can additionally be embedded in the image using the present dotorientation method which provide the decoder process with a list of theunique states the embedded single data values can take and provideinformation about which unique dot orientation directions are associatedwith each of the unique states. Other information about the size of theperiodic tiles can also be embedded. In this embodiment, the decoderwould not need to know any prior information about the tiling,orientation directions, and associated unique states used to embed thedata message in the image. The decoder would only have to know about thetags and their locations.

In addition to the many features and advantages previously mentioned,the halftone encoding process can be performed offline for eachclustered dot design to be supported by a particular printer. Forexample, the conversion can be performed by a software-based embodimentimplemented in a computer system or a control unit in communication withthe image path of a xerographic or other document production device.Further, the present method for data embedding via halftone dotorientation is well matched to the characteristics of halftone printingand causes only a minimal increase in the distortion pre-existing inhalftones. This is because the halftone reproduction process relies onmatching the average gray-level of the print to the desired image andthe change in dot orientations induced by dot orientation modulationembedding does not change the average gray level within each halftonecell.

For typical printing resolutions, halftone dot orientation-basedembedding causes minimal degradation in output image quality. When thetwo orthogonal orientations used for the embedding are symmetricallylocated with respect to the two orthogonal directions that defineprinter addressability, the symmetry in the printing system ensures thatthe tone response for halftones of the two orientations is nearidentical. Thus, embedding does not change the average gray level, whichis one fundamental image fidelity criterion in halftone reproduction.This happens for instance when the embedding utilizes a +/−45°orientation in a printing system with the natural 0°/90° axis.

What is also disclosed herein is a method for decoding data from ascanned image print containing the original message embedded via theabove described dot orientation method.

Reference is now made to FIG. 9 which broadly illustrates one exampleblock diagram for decoding data from a scanned image which was embeddedusing the present halftone dot orientation method. The halftoned image902 containing the oriented elliptical halftone dots in accordance withthe present method is scanned using scanning device 904. The scanning ofthe print produces scanned image 906, denoted I^(s)(x,y). The scannedimage comprises a plurality of color data points. The color data is thenlocally and globally synchronized. For screened halftones (bothclustered and dispersed-dot), the inherent periodicity of the screeningprocess provides a natural template for global synchronization. Peaks inthe Fourier Transform 908 of the scanned image are used to estimateglobal rotation and scaling factors. At 910, moment extraction and localimage gray level estimation is performed. The extracted moments and graylevel estimations are provided to a channel decoder 912. Since theoriginal message is embedded in oriented halftone dots, statisticalcriteria that capture dot orientation are used by the channel decoderfor data extraction. The extracted message 914 is retrieved.

Reference is now made to the flow diagram of FIG. 10 which illustratesone example embodiment of the present method for decoding data which hasbeen embedded via halftone dot orientation.

At 1002, a printed image containing an encoded message is scanned. Themessage encoded in the image comprises a plurality of single datavalues. Each single data value assumes only one unique state from theset of binary values used for encoding. Each of the unique states isassociated with a unique orientation direction. At 1004, the scannedimage is synchronized to compensate for geometric distortions introducedinto the image by the print/scan process. In one embodiment, the imageis synchronized based on estimated global rotation and scaling factorscomputing from a probabilistic model of the scan channel of the scanningdevice used to scan the print image. At 1006, the scanned image ispartitioned into a periodic tiling. Each of the uniform periodic tilesencompasses a single halftone dot. The uniform tiling has the sameperiodicity as that used when the original message was embedded in theimage. At 1008, for each of the periodic tiles in the scanned image, aplurality of image moments are calculated for the halftone dot of thecurrent tile. One image moment is calculated along each of the uniqueorientation directions. Image moments are based on a computed center ofmass for the halftone dot. An orientation direction for the halftone dotin the current tile is determined based upon the largest calculatedimage moment. All pixels in a particular halftone cell contribute tomoments along their respective orientation direction. The largerdistance to the center of mass (along the respective axis) isinterpreted to mean that the dot is elongated along that axis and thusoriented in that direction. The identified orientation direction isassociated with one of the unique states a single data value can take.The single data value embedded in the current tile is decoded based onthe unique state associated with the determined orientation direction.The collection off decoded single data values forms the originalmessage. At 1010, the message embedded in the printed image isreassembled from the decoded single data values.

In an alternative embodiment, the original message is encoded in one ormore specific regions of the contone image. In this case, regions oftiles containing embedded single data values would have to be firstidentified. As previously discussed with respect to the data encodingmethod, one or more “tags” are placed within the image at predeterminedtile locations. The tags identify regions in the image where theoriginal message is located. Such tags in predetermined regions of theimage would first be decoded. The decoded tags would indicate one ormore regions within the image where the original message is located.Those unique tiles wherein the original message is embedded would bedecoded. The single data values decoded from these identified tileswould form the original message. Other variations are intended to fallwithin the scope of the appended claims.

Embodiments for calculating image moments (σ_(x),σ_(y)) for uniqueorientation directions based on a computed center of mass for a givendot will next be discussed.

Dot orientation estimation is based on statistically motivated imagemoments computed within each halftone cell. In one embodiment, the imagemoment along the x-axis is determined by:

$\sigma_{x} = {\frac{1}{S}{\sum\limits_{x,{y \in C}}{{I^{s}( {x,y} )}( {x - \overset{\_}{x}} )^{2}}}}$

where

$S = {{\sum\limits_{x,{y \in C}}{{I^{s}( {x,y} )}\mspace{14mu}{and}\mspace{14mu}\overset{\_}{x}}} = {\frac{1}{s}{\sum\limits_{x,{y \in C}}{{I^{s}( {x,y} )}x}}}}$represents the abscissa of the center of mass of the halftone dot andthe range of the summations C denotes the spatial extent of the halftonecell corresponding to the halftone dot. The moment σ_(y) along they-axis is calculated in a similar fashion. The moment vector, given by(σ_(x), σ_(y)), is interpreted as a lower-dimensional (2D) featurevector that identifies orientation.

Thus:

$\sigma_{x} = \frac{\sum\limits_{x,{y \in C}}{{I^{s}( {x,y} )}( {x - \overset{\_}{x}} )^{2}}}{\sum\limits_{x,{y \in C}}{I^{s}( {x,y} )}}$$\sigma_{y} = \frac{\sum\limits_{x,{y \in C}}{{I^{s}( {x,y} )}( {y - \overset{\_}{y}} )^{2}}}{\sum\limits_{x,{y \in C}}{I^{s}( {x,y} )}}$

where:

$\overset{\_}{x} = {{\frac{\sum\limits_{x,{y \in C}}{{I^{s}( {x,y} )}x}}{\sum\limits_{x,{y \in C}}{I^{s}( {x,y} )}}\mspace{14mu}{and}\mspace{14mu}\overset{\_}{y}} = \frac{\sum\limits_{x,{y \in C}}{{I^{s}( {x,y} )}y}}{\sum\limits_{x,{y \in C}}{I^{s}( {x,y} )}}}$

represents the center of mass of the halftone dot, and σ_(x), σ_(y)indicate image moments calculated along orthogonal x and y axis,respectively. It is understood that the above equations represent thedesired image moments for the case where less than half of the halftonetile is covered by toner or ink in the marking process. In theapplication of the method for regions where more than half of thehalftone tile is covered by toner or ink, the scanned image data isinverted prior to the computation of the moments. It will also bereadily apparent to those skilled in the art that if more than twoorientations are utilized for the embedding, moments corresponding tothose orientations can be readily computed through a simple rotation ofthe coordinate axes to the desired orientation prior to computing themoments.

Decoding using the calculated image moments along each respectiveorientation direction involves determining a larger of the calculatedmoments. One such rule for the case of two embedding orientations isgiven by the relationship:

$\sigma_{x}\overset{1}{\underset{0}{\gtrless}}{\sigma_{y}.}$

If the calculated moment in the x direction is larger than they-direction, for example, then the dot is determined to be orientedpredominantly along the x-axis and the resulting associated unique stateis 1. If, on the other hand, the calculated moment is larger in they-direction then the dot is oriented predominantly in the y-directionand the resulting associated unique state is 0. The single data valueembedded in this tile would be decoded by the unique state determined bythe unique state associated with the orientation direction determined bythe direction of the largest calculated moment.

The moments along the orthogonal (x,y) axis can be seen as beingapproximately conditionally independent given the embedding orientationand the average gray level.

For channel coding, two code families, i.e., convolutional codes andrepeat accumulate (RA) codes are considered. One skilled in this artwould be familiar with both code families. See: “Coding theorems for‘turbo-like’ codes”, D. Divsalar, H. Jin, R. J. McEliece, Proc. AllertonConference, 201-210, Monticello, Ill., USA, (September 1998), which isincorporated herein in its entirety by reference. See also: “IrregularRepeat-Accumulate Codes”, H. Jin, A. Khandekar, R. McEliece, Proc. 2ndInt'l. Conference on Turbo Codes, Brest, France, (September 2000), whichis incorporated herein in its entirety by reference. The choice ofconvolutional codes is motivated by the fact that good codes with low tomoderate complexity are known for a variety of rates and the merit ofthe probabilistic channel model can be clearly demonstrated for thesecodes by contrasting soft and hard decoding. The choice of RA codes wasmotivated by the fact that these are near capacity-achieving and alsoreadily allow the rate to be varied through a change of the repetitionparameter. Maximum likelihood decoding of convolutional codes can beachieved by, for instance, utilizing a Viterbi algorithm which allowsfor both hard and soft decoding. Approximate maximum a posterioriprobability (MAP) decoding for RA codes is accomplished using beliefpropagation for iterative decoding.

For both convolutional and RA codes, decoding used a log likelihoodratio:

$\gamma = {\log\;\frac{p( {\Theta_{1},\hat{g}} )}{p( {\Theta_{2},\hat{g}} )}}$

where p(⊖_(i,^g)) is the likelihood given by:p(⊖_(i) ,ĝ)=f _(xy)(σ_(x)|⊖_(i) ,ĝ)·f_(xy)(σ_(y)|⊖_(i) ,ĝ)

where σ_(x), σ_(y) are the image moments along the x and y directionscomputed within the halftone cell and ^g is the estimate of the localgray-level obtained as the average of the pixel intensity values withinthe halftone cell.

For the convolutional codes, the use of the image adaptive decoding withthe channel characterization yields much lower bit error rates than thehard decoding alternative which does not utilize this characterization.For the RA codes, the benefit of utilizing embedding schemes anddecoding methods that are specifically designed for the print-scanchannel in consideration—(operational) error-free embedding ratesachieved are much greater than those reported by existing hardcopy datahiding methods.

In order to incorporate channel dependence on the local gray level ofthe cover image, a statistical model for the channel needs to beexpressed in the form of a conditional density function, f(σ_(x),σ_(y)/⊖_(i), g). The received image moments are conditioned on the localimage gray level g and, horizontal and vertical orientations given by:⊖_(i,i)=1,2.

A probabilistic model can be constructed which conditions receivedmoments on input orientations by assuming conditional independence forthe multi-dimensional density functions. One such a model is given by:

where the following conditional independence is assumed:f _(σ) _(x) ,_(σ) _(y) (σ_(x),σ_(y)|⊖_(g))=f _(σ) _(x) (σ_(x)|⊖_(g))·f_(σ) _(y) (σ_(y)|⊖_(g)).

The above assumption can be validated by observing 2×2 covariancematrices of the received random vector a conditioned on ⊖₁,⊖₂, forseveral gray levels. Empirical estimates of these covariance matricesobtained from scanned images corresponding to prints for different graylevels are provided. It should be appreciated that most of the matricesare close to diagonal thereby indicating that they may approximately bemodeled as independent (because zero covariance across the off-diagonalsguarantees “uncorrelatedness” and not independence). One examplecovariance matrix is as follows:

$C_{1} = \begin{bmatrix}0.36 & 0.08 \\0.08 & 0.17\end{bmatrix}$ $C_{2} = \begin{bmatrix}0.54 & 0.02 \\0.02 & 0.26\end{bmatrix}$ $C_{3} = \begin{bmatrix}0.56 & 0.07 \\0.07 & 0.29\end{bmatrix}$ $C_{4} = \begin{bmatrix}0.80 & 0.07 \\0.07 & 0.53\end{bmatrix}$ $C_{5} = \begin{bmatrix}1.02 & {- 0.20} \\{- 0.20} & 1.21\end{bmatrix}$ $C_{6} = \begin{bmatrix}0.20 & 0.07 \\0.07 & 0.40\end{bmatrix}$ $C_{7} = \begin{bmatrix}0.22 & 0.07 \\0.07 & 0.40\end{bmatrix}$ $C_{8} = \begin{bmatrix}0.26 & 0.01 \\0.01 & 0.56\end{bmatrix}$ $C_{9} = \begin{bmatrix}0.39 & 0.06 \\0.06 & 0.70\end{bmatrix}$ $C_{10} = \begin{bmatrix}0.50 & {- 0.09} \\{- 0.09} & 0.82\end{bmatrix}$

Conditional densities can be characterized in terms of experimentallydetermined empirical histograms. FIG. 11 shows the normalized histogramsof the received moments σ_(x), σ_(y). FIG. 11A shows histograms of thereceived moments σ_(x), σ_(y) for the (transmitted) horizontalorientation ⊖₁ given two orientations at a gray level of 32%. FIG. 11Bshows the histograms of σ_(x), σ_(y) for the vertical orientation ⊖₂given two orientations at a gray level of 32%. FIGS. 11C-D repeat thehistograms of FIGS. 11A-B for a gray level of 52%. The conditionaldensities were modeled by the exponential power density familyparameterized by the scale parameter a and shape parameter b. Oneembodiment of an exponential power density is given by:

${f_{X}(x)} = {\frac{1}{2a\;{\Gamma( {1 + \frac{1}{b}} )}}{\mathbb{e}}^{({- {\frac{\pi}{a}}^{b}})}}$

For b=1, this reduces to a Laplacian distribution. For b=2, this takes aGaussian form. The parameters of the distribution are estimated usingthe Expectation-Maximization algorithm for exponential families. See:“The expectation-maximization algorithm”, T. K. Moon, IEEE SignalProcessing Magazine, Vol. 13, No. 6, 47-60, (1996), which isincorporated herein in its entirety by reference.

A decision on dot orientation can then be made using amaximum-likelihood criterion. One example criterion is given by:

$\Theta^{s} = {\arg\;{\max\limits_{{i = 1},2}{{f_{\sigma_{x}}( \sigma_{x} \middle| \Theta_{i} )} \cdot {f_{\sigma_{y}}( \sigma_{y} \middle| \Theta_{i} )}}}}$

Orientation parameters were set such that: γ_(x)=1 and γ_(y)=2, in orderto achieve a horizontally oriented dot. The parameters are swapped toorient the dot in a vertical direction. The halftone images weregenerated with a screen frequency of 75 cells per inch (cpi) along bothvertical and horizontal directions. If each halftone cell is used forembedding, this amounts to 360,000 bits embedded in the image.

TABLE 1 enumerates various bit error rate (BER) performances fordifferent gray levels with hard decoding and Maximum-Likelihood (ML)decoding. The term “NS” is used to indicate that symbol synchronizationis not achieved for the corresponding gray level.

TABLE 1 Gray Level (percent) 10 20 30 40 50 60 70 80 Hard NS 0.01090.0032 0.0238 NS 0.0045 0.0303 NS Decoding BER ML NS 0.0085 0.00120.0073 NS 0.0043 0.0339 NS Decoding BER

As is evident from TABLE 1, data embedded in gray level areas rangingfrom 20% to 40% and from 60% to 70% can be recovered with a small BER.Note that 0% gray level corresponds to digital level 255 and 100% graylevel corresponds to digital level 0. In highlights, shadows andmid-tone gray levels, however detection was not possible because symbolsynchronization is completely lost in those regions.

Reference is now made to FIG. 12 which plots BER performance (as inTABLE 1) but across a larger number of gray levels. The plot shows thatthe lowest BER is achieved for gray levels around 30% and 60%. This isintuitively clear as highlights, shadows, and mid-tones (approx. 50%) donot provide sufficiently robust dot configurations. Thus, goodperformance can be expected between white and mid-tones, and black andmid-tones. A counter-intuitive observation is that at the 70% graylevel, soft or Maximum-Likelihood (ML) decoding performs marginally lessgood than does hard decoding. This can be attributed to deviations ofthe assumed channel model from the true one and the inability tosynchronize for gray levels in the 70% range.

Reference is now made to FIG. 13 which illustrates decoding results inthe form of bit error rate (BER) as a function of code rate. As shown,the cover dependent channel model reduces bit errors. This benefit isparticularly pronounced at low code rates. As with convolutional codes,several different halftone images were generated with embedded datagenerated using RA coding on message bits. From scans of these printedimages, it was determined that a code rate of ¼ allows for error freedecoding. Hence, this rate provides a lower bound on data hidingcapacity, i.e., 0.25 bits per halftone cell. This rate exceeds existingdata embedding methods and approaches the capacity of hardcopy dataencoding methods known in this art.

Reference is now made to FIG. 14 which illustrates the effects of thepresent data embedding method on a sample cover image, shown at FIG.14A. FIGS. 14B-E show enlarged local views from a scan of a print of theimage generated using the above described dot orientation method. Theregions of the cover image, to which the associated local viewscorrespond, are indicated by the corresponding labels. Ellipticallyshaped dots are utilized with a vertical/horizontal orientation along amajor axis depending on the binary output value for each pixel in eachtile. Black elliptical dots along the two orientations are produced on apredominantly white background in highlight areas, i.e. regions of thecover image with less than 50% halftone area coverage (FIG. 14B). Whiteelliptical “holes” with the two possible orientations are produced inshadow areas, i.e. regions of the image with more than 50% halftone areacoverage (FIG. 14C). While the embedding distortion characteristics aregood, due to the inherent constraints of halftone reproduction, therobustness of the present dot orientation method demonstratessignificant spatial variability depending on the local content of thecontone image. In regions where the image is white, there is no printingand hence the computed image moments have no discernible dependence onthe embedded single data value. This limitation also arises in regionswhere the cover image is close to white (FIG. 14D). In regions where thecover image is black, the halftone cells in the print are completelyblack. Consequently, the moments at the receiver convey no usefulinformation about the embedded bit in regions where the cover image isblack or near black. In mid-gray regions of the cover image thatcorrespond to 50% halftone area coverage, and in regions where the areacoverage is close to 50%, the printed halftones have the configurationof a checker-board pattern and orientation modulation is ineffective(FIG. 14E).

The present method was evaluated utilizing a print-scan channelconsisting of a xerographic printer and a desktop scanner. The coverimage dependence of the channel was characterized and decodingperformance evaluated over test prints. A randomly generated bit streamwas used as the original message to avoid any prescience. The data wasencoded utilizing the encoder for the error correction code inconsideration of either convolutional or RA codes. The resulting codeddata and a target cover image served as the input to a module whichjointly performed the halftoning and the data embedding operations. A45° orthogonal halftone screen with a frequency of 75 cells per inch wasutilized. The orientation of the screen should not be confused with theorientation of the halftone dots used for embedding. The resultinghalftone image (with data embedded in the orientation) was printed on acolor marking device which had an addressability of 2400 dpi. Theresulting print containing the embedded data message was in turn scannedusing a desktop scanner with a resolution of 1200 dpi. Globalsynchronization was performed on the scanned image utilizing the definedperiodic grid of the halftone cells. Geometric distortion in theprinting process was accounted for via decision directed localsynchronization. Post-synchronization, the estimate ^g of the averagegray-level in the halftone cell was obtained. Image moments σ_(x) andσ_(y) along the x and y axis, respectively, were computed in the mannerdiscussed above.

The print-scan channel was characterized utilizing a set of trainingimages from which the parameters for the statistical channel model wereobtained via the Expectation-Maximization algorithm. System performancewas then evaluated utilizing an independent test image. The printed sizeof the image was 8″×8″. A total of 414,960 coded bits were embedded inthe printed image. When using convolutional codes, the coded bits andcorresponding halftone images with data embedding were generated forseveral different code rates 1/n, where n={2, 3, 4, 5, 6, 7, 8, 10, 12}.Convolutional codes used are known codes with low constraint length.

Results are further discussed in: “Data Embedding in Hardcopy Images ViaHalftone-Dot Orientation Modulation”, O. Bulan, V. Monga, G. Sharma, andB. Oztan, Proc. SPIE: Security, Forensics, Steganography, andWatermarking of Multimedia Contents X Electronic Imaging Symp., SanJose, Calif., USA (Jan. 27-31, 2008), which is incorporated herein inits entirety by reference. For a discussion regarding capacity fororientation modulation channels for data embedding in printed halftones,see: “On the Capacity of Orientation Modulation Halftone Channels”, O.Bulan, V. Monga, and G. Sharma, Proc. IEEE Intl. Conf. Acoustics Speechand Sig. Proc., pp. 1685-1688, Las Vegas, Nev., USA (March 30-Apr. 4,2008), which is incorporated herein in its entirety by reference. Thispaper discusses capacity upper bounds for orientation modulationchannels and reveals that the channel capacity shows considerablevariation as a function of gray level. This result helps in identifyinghiding friendly gray levels.

Reference is now made to FIG. 15 which illustrates one example documentreproduction system wherein various aspects of the present method arelikely to find their intended uses. Such an example documentreproduction system has a scanner capable of scanning the cover contoneimage to obtain pixel intensity values. Arrays of threshold values arestored in memory or retrieved from a storage device. A special purposecomputer or software loaded thereon would perform the calculationsdescribed herein to determine the orientation directions of each of thehalftone dots of each periodic cell wherein a single data value of theoriginal message is intended to be embedded in the halftone image. Sucha system would be capable of rendering an output image containing theembedded data message. Such an example document reproduction would alsobe capable of receiving an image containing a data message embedded viahalftone dot orientation through a document scanning device or from astorage device or from a network. The scanning device incorporated withsuch a document reproduction system would have sufficient resolution toscan the individual pixels necessary to obtain the data required for thecalculations described above to determine the orientation of thehalftone dots of each of the periodic cells wherein a single data valuehas been encoded. The system would determined the unique state of eachof the encoded single data values based on the determined halftone dotorientations and retrieve the original message embedded in the scannedimage print. The present method could also readily find its uses in aworkstation environment similarly configured.

The example document reproduction system 1500 includes an input module1502 having a controller 1504, a document processing module 1506, and aprinter module 1508. The document processing module includes a documentprocessing station 1510 that includes a display 1512, a keyboard 1514,and a mouse 1518. A scanning device 1520 is used to scan the imagepixels of a color image. The document processing station may be used byan operator to set parameters or scan operations and other documentprocessing and document printing operations. The instructions for thesevarious operations may be input via the keyboard and/or mouse, or touchscreen objects displayed on the display. The document processing stationalso includes a processor having memory and secondary storage, such as adisk drive, for storage of programs and data required for processingdocuments through system 1100. The printer module 1108 also includesprocessors and controllers for regulating the application of inks ortoners onto paper as well as the control of papers moving through theprinter module for proper registration in multi-channel color printing,and the like. The system includes one or more discharge areas 1122 wherefinished documents are deposited for retrieval. The special purposecontroller 1104 (internal to system 1100) monitors and regulates theoperation of the scanning device 1120 for obtaining the image pixel datapoints of the color image containing the original message embedded bythe method described above. The controller includes a processor capableof executing machine program instructions for carrying out variousaspects of the present method via any of a micro-processor,micro-controller, ASIC, electronic circuit, or special purpose computeras described herein further. Portions of the flow diagrams of thepresent method may also be implemented partially or fully in hardware inconjunction with machine executable instructions in communication withthe controller. The controller also includes a network connection (notshown) for receiving color data points over a network such as anintranet or internet. The controller may also be in digitalcommunication with one or more electronic media readers for the input ofimage data from electronic storage media such as, for example, a CD-ROM,or magnetic disk.

One or more aspects of the present method can be implemented on aspecial purpose computer. Such a special purpose computer can beintegrated, in whole or in part, with, for example, a xerographic systemor a color management or image processing software or computing system,which includes a processor capable of executing machine readable programinstructions for carrying out one or more aspects of the present method.Such a processor is in communication with a bus (e.g., a backplaneinterface bus, cross-over bar, or data network). The system includes amain memory capable of storing machine readable instructions to beexecuted and may include random access memory (RAM) to supportreprogramming and flexible data storage. The main memory may furtherinclude one or more buffers to store data. Memory may further be used tostore executable machine program instructions that implement the methodsdescribed herein. The computer system may also include a secondarymemory. The secondary memory may include, for example, a hard disk driveand/or a removable storage drive which reads/writes to a removablestorage device such as a floppy disk, magnetic tape, optical disk, etc.,used to store computer software and other machine readable instructions,and/or data. The secondary memory may also include other mechanisms forallowing computer programs or other instructions to be loaded into thecomputer system for execution. Such mechanisms may include, for example,removable storage adapted to exchange data through an interface.Examples of such mechanisms may further include a program cartridge andcartridge interface such as that found in video game devices, aremovable memory chip such as an EPROM, or PROM, and associated socket,and other removable storage units and interfaces which allow softwareand/or data to be transferred to the computer system.

The computer system may additionally include a communications interfacewhich acts as both an input and an output to allow software and data tobe transferred between the computer system and external devices.Examples of a communications interface include a modem, a networkinterface such as an Ethernet card, a communications port, a PCMCIA slotand card, etc. Software and data transferred via the communicationsinterface are in the form of signals and data which may be, for example,electronic, electromagnetic, optical, or other signals capable of beingtransmitted and received via a communications path (i.e., channel)configured to carry such signals and may be include wire, cable, fiberoptic, phone line, cellular link, RF, or other communications channels.The computer system may additionally include a display interface thatforwards data from a communication bus (or from a frame buffer) to adisplay device. The computer system may further include a scanningdevice capable of receiving color images and transforming the imagesinto an electronic format. Such a scanning device would be capable ofdetermining the color, location, and other information regarding theimage pixels contained in the scanned color image.

Reference is now made to FIG. 16 which is an explanatory diagramillustrating one example of a computer readable storage medium capableof storing machine readable instructions which, when mounted on acomputer system, cause the computer system to perform one or moreaspects of the present method as described herein above. The machinereadable instructions may be modified by one computer system andtransferred to another computer system. One or more computer programinstructions 1600 for carrying out the present method are loaded oncomputer-readable storage media 1602 which includes media such asoptical disks (CD-ROM etc.), magnetic disks, magnetic cards, memories(including IC cards and memory card). The storage media stores themachine readable program instructions for sale, transport, and storageby changing magnetic, optical, and/or electric energy states in responseto program description instructions having been transferred to themedia. The storage medium can then be mounted on computer system 1604and transferred or otherwise communicated to computer system 1606. Theprogram instructions can then be off-loaded to another program 1608, inoriginal form or modified, including data, and stored on storage media1610. Both of the computer systems include processors capable ofexecuting machine readable program instructions.

Terms such as, computer program medium, computer readable medium,computer executable medium, and computer usable medium are used hereinto generally refer to a machine readable media such as main memory,secondary memory, removable storage device such as a hard disk, andcommunication signals. Such computer program products are means forproviding instructions and/or data to the computer system or device forimplementing the present method. The computer readable medium storesdata, instructions, messages packets, or other machine readableinformation. The computer readable medium may include non-volatilememory, such as a floppy disk, hard drive, memory, ROM, RAM, flashmemory, disk memory, and other permanent storage useful, for example,for transporting information such as data and machine readable programinstructions. It may further include a CD-ROM, DVD, tape, cassette, orother digital or analog media, capable of having embodied thereon one ormore logical programming instructions or other machine executable codesor commands that implement and facilitate the function, capability, andmethods disclosed herein. The computer readable medium may additionallycomprise information in a transitory state medium such as a network linkor a network interface which may include a wired network or a wirelessnetwork which allows a computer to read such computer readableinformation.

It should be understood that the flow diagrams of the present method areillustrative. Other operations, for example, may be added, modified,enhanced, condensed, integrated, or consolidated. Variations thereof areintended to fall within the scope of the appended claims. It should alsobe understood that one or more aspects of the present method areintended to be incorporated in an article of manufacture, including oneor more computer program products. The article of manufacture may beincluded on at least one storage device readable by a machinearchitecture, xerographic, color management, or other image processingsystem capable of executing program instructions. The article ofmanufacture may be included as part of a xerographic system, colormanagement system, an operating system, a software program, a plug-in.Such an article of manufacture may further be shipped, sold, leased, orotherwise provided separately either alone or as part of an add-on,update, upgrade, or product suite by the present assignee or a licenseethereof.

It will be appreciated that the above-disclosed features and functionsand variations thereof may be desirably combined into many otherdifferent systems or applications. Various presently unforeseen orun-anticipated alternatives, modifications, variations, or improvementsmay become apparent and/or subsequently made by those skilled in the artwhich are also intended to be encompassed by the appended claims. Theembodiments set forth above are considered to be illustrative and notlimiting. Various changes to the above-described embodiments may be madewithout departing from the spirit and scope of the invention.

1. A method for decoding data embedded in an image via halftone dotorientation, the method comprising: scanning an image print containing adata message embedded therein, said message comprising a plurality ofsingle data values, each single data value taking only one unique state;synchronizing said scanned image to a uniform periodic tiling utilizedfor generating said image print to compensate for geometric distortionsintroduced by print/scan processes performed on said image print;associating a unique orientation direction with each of said uniquestates, each of said unique orientation directions being oriented withrespect to a horizontal and vertical axis of said scanned image print;defining a uniform periodic tiling for said scanned image; and for eachof said uniform periodic tiles in said uniform tiling, calculating aplurality of image moments for a halftone dot of a current tile, oneimage moment being calculated along each of said unique orientationdirections; determining a largest of said calculated moments, saidlargest moment identifying a dominant orientation direction for saidhalftone dot; and decoding a single data value embedded in said currenttile based on which of said unique states is associated said determineddominant orientation direction.
 2. The method of claim 1, whereinsynchronizing said scanned image comprises: obtaining a frequency domainrepresentation of image data obtained from said scanned image print;locating positions of magnitude peaks in said scanned data; estimating aglobal rotation and a scaling factor from said magnitude peak positions;and synchronizing said scanned image based on said estimated globalrotation and scaling factor.
 3. The method of claim 1, whereinsynchronizing said scanned image further comprises estimating an extentof each uniform tile in the scan coordinate space based on scan data ofneighboring uniform tiles, said neighbor tiles preceding said currenttile in the processing sequence.
 4. The method of claim 1, whereincalculating two of said image moments (σ_(x), σ_(y)) for uniqueorientation directions along orthogonal x and y axis comprises:$\sigma_{x} = \frac{\sum\limits_{x,y,{\in C}}{{I^{s}( {x,y} )}( {x - \overset{\_}{x}} )^{2}}}{\sum\limits_{x,{y \in C}}{I^{s}( {x,y} )}}$$\sigma_{y} = \frac{\sum\limits_{x,{y \in C}}{{I^{s}( {x,y} )}( {y - \overset{\_}{y}} )^{2}}}{\sum\limits_{x,{y \in C}}{I^{s}( {x,y} )}}$where:$\overset{\_}{x} = {{\frac{\sum\limits_{x,{y \in C}}{{I^{s}( {x,y} )}x}}{\sum\limits_{x,{y \in C}}{I^{s}( {x,y} )}}\mspace{14mu}{and}\mspace{14mu}\overset{\_}{y}} = \frac{\sum\limits_{x,{y \in C}}{{I^{s}( {x,y} )}y}}{\sum\limits_{x,{y \in C}}{I^{s}( {x,y} )}}}$represents a center of mass of said halftone dot along each axisrespectively and C denotes a spatial extent of the halftone tile.
 5. Themethod in claim 4, wherein synchronizing said scanned image furthercomprises: for each tile in said uniform periodic tiling, estimating acenter of said tile based on a center of mass determined for a halftonedot in said current tile; and using center of mass coordinates for saidcurrent tile and center of mass coordinates of spatially adjacentneighboring tiles surrounding said current tile to estimate a region inscan coordinates corresponding to a next tile to be processed.
 6. Themethod of claim 4, further comprising calculating image moments alongadditional orientations obtained through a rotation of the x-ycoordinate axis.
 7. The method in claim 1, further comprising computinga plurality of image moments over a plurality of tiles in an errorcorrection decoder to estimate the message embedded in said image print.8. The method of claim 1, further comprising retrieving a tag from apredetermined location in said scanned image, said tag indicating astarting tile location in said image wherein said message data can beretrieved.
 9. The method of claim 1, further comprising performing anerror correction on said decoded message.
 10. A system for decoding dataembedded in an image via halftone dot orientation, the systemcomprising: a storage medium capable of storing data; and a processor incommunication with said storage medium, said processor capable ofexecuting a machine readable instruction for performing the method of:scanning an image print containing a data message embedded therein, saidmessage comprising a plurality of single data values, each single datavalue taking only one unique state; synchronizing said scanned image toa uniform periodic tiling utilized for generating said image print tocompensate for geometric distortions introduced by print/scan processesperformed on said image print; associating a unique orientationdirection with each of said unique states, each of said uniqueorientation directions being oriented with respect to a horizontal andvertical axis of said scanned image print; defining a uniform periodictiling for said scanned image; and for each of said uniform periodictiles in said uniform tiling, calculating a plurality of image momentsfor a halftone dot of a current tile, one image moment being calculatedalong each of said unique orientation directions; determining a largestof said calculated moments, said largest moment identifying a dominantorientation direction for said halftone dot; and decoding a single datavalue embedded in said current tile based on which of said unique statesis associated said determined dominant orientation direction.
 11. Thesystem of claim 10, wherein synchronizing said scanned image comprises:obtaining a frequency domain representation of image data obtained fromsaid scanned image print; locating positions of magnitude peaks in saidscanned data; estimating a global rotation and a scaling factor fromsaid magnitude peak positions; and synchronizing said scanned imagebased on said estimated global rotation and scaling factor.
 12. Thesystem of claim 10, wherein calculating two of said image moments(σ_(x),σ_(y)) for unique orientation directions along orthogonal x and yaxis comprises:$\sigma_{x} = \frac{\sum\limits_{x,{y \in C}}{{I^{s}( {x,y} )}( {x - \overset{\_}{x}} )^{2}}}{\sum\limits_{x,{y \in C}}{I^{s}( {x,y} )}}$$\sigma_{y} = \frac{\sum\limits_{x,{y \in C}}{{I^{s}( {x,y} )}( {y - \overset{\_}{y}} )^{2}}}{\sum\limits_{x,{y \in C}}{I^{s}( {x,y} )}}$where:$\overset{\_}{x} = {{\frac{\sum\limits_{x,{y \in C}}{{I^{s}( {x,y} )}x}}{\sum\limits_{x,{y \in C}}{I^{s}( {x,y} )}}\mspace{14mu}{and}\mspace{14mu}\overset{\_}{y}} = \frac{\sum\limits_{x,{y \in C}}{{I^{s}( {x,y} )}y}}{\sum\limits_{x,{y \in C}}{I^{s}( {x,y} )}}}$represent a center of mass of said halftone dot along each axisrespectively and C denotes a spatial extent of the halftone tile. 13.The system in claim 12, wherein synchronizing said scanned image furthercomprises: for each tile in said uniform periodic tiling, estimating acenter of said tile based on a center of mass determined for a halftonedot in said current tile; and using center of mass coordinates for saidcurrent tile and center of mass coordinates of spatially adjacentneighboring tiles surrounding said current tile to estimate a region inscan coordinates corresponding to a next tile to be processed.
 14. Thesystem of claim 12, further comprising calculating image moments alongadditional orientations obtained through a rotation of the x-ycoordinate axis.
 15. The system in claim 10, further comprisingcomputing a plurality of image moments over a plurality of tiles in anerror correction decoder to estimate the message embedded in said imageprint.
 16. The system of claim 10, further comprising performing anerror correction on said decoded message.
 17. A computer program productfor decoding data embedded in an image via halftone dot orientation, thecomputer program product comprising: a non-transitory computer readablemedia for storing instructions that, when executed on a computer, causethe computer to perform a method comprising: scanning an image printcontaining a data message embedded therein, said message comprising aplurality of single data values, each single data value taking only oneunique state; synchronizing said scanned image to a uniform periodictiling utilized for generating said image print to compensate forgeometric distortions introduced by print/scan processes performed onsaid image print; associating a unique orientation direction with eachof said unique states, each of said unique orientation directions beingoriented with respect to a horizontal and vertical axis of said scannedimage print; defining a uniform periodic tiling for said scanned image;and for each of said uniform periodic tiles in said uniform tiling,calculating a plurality of image moments for a halftone dot of a currenttile, one image moment being calculated along each of said uniqueorientation directions; determining a largest of said calculatedmoments, said largest moment identifying a dominant orientationdirection for said halftone dot; and decoding a single data valueembedded in said current tile based on which of said unique states isassociated said determined dominant orientation direction.
 18. Thecomputer program product of claim 17, wherein calculating two of saidimage moments (σ_(x), σ_(y)) for unique orientation directions alongorthogonal x and y axis comprises:$\sigma_{x} = \frac{\sum\limits_{x,{y \in C}}{{I^{s}( {x,y} )}( {x - \overset{\_}{x}} )^{2}}}{\sum\limits_{x,{y \in C}}{I^{s}( {x,y} )}}$$\sigma_{y} = \frac{\sum\limits_{x,{y \in C}}{{I^{s}( {x,y} )}( {y - \overset{\_}{y}} )^{2}}}{\sum\limits_{x,{y \in C}}{I^{s}( {x,y} )}}$where:$\overset{\_}{x} = {{\frac{\sum\limits_{x,{y \in C}}{{I^{s}( {x,y} )}x}}{\sum\limits_{x,{y \in C}}{I^{s}( {x,y} )}}\mspace{14mu}{and}\mspace{14mu}\overset{\_}{y}} = \frac{\sum\limits_{x,{y \in C}}{{I^{s}( {x,y} )}y}}{\sum\limits_{x,{y \in C}}{I^{s}( {x,y} )}}}$represent a center of mass of said halftone dot along each axisrespectively and C denotes a spatial extent of the halftone tile. 19.The computer program product in claim 17, wherein synchronizing saidscanned image comprises: obtaining a frequency domain representation ofimage data obtained from said scanned image print; locating positions ofmagnitude peaks in said scanned data; estimating a global rotation and ascaling factor from said magnitude peak positions; and synchronizingsaid scanned image based on said estimated global rotation and scalingfactor.
 20. The computer program product in claim 17, further comprisingperforming an error correction on said decoded message.